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Abstract. The aim of this work is to extend the study of super-coinvariant poly- 
nomials, introduced in [21 [5, to the case of the generalized symmetric group Gn,m, 
defined as the wreath product Cm } 5„ of the symmetric group by the cyclic group. 
We define a quasi-symmetrizing action of Gn,m on Q[a;i, . . . , x„], analogous to those 
defined in |12| in the case of iS„. The polynomials invariant under this action are 
called quasi- invariant, and we define super-coinvariant polynomials as polynomials 
orthogonal, with respect to a given scalar product, to the quasi-invariant polynomi- 
als with no constant term. Our main result is the description of a Grobner basis for 
the ideal generated by quasi- invariant polynomials, from which we dedece that the 
dimension of the space of super-coinvariant polynomials is equal to m" C„ where 
Cn is the n-th Catalan number. 

Resume. Le but de ce travail est d'etendre I'etude des polynomes super-coinvariants 
(definis dans [5]), au cas du groupe symetrique generalise Gn,m, defini comme le 
produit en couronne Cm I Sn du groupe symetrique par le groupe cyclique. Nous 
definissons ici une action quasi-symetrisante de Gn,m sur Q[a;i, . . . , Xn], analogue a 
celle definie dans [12j dans le cas de Sn- Les polynomes invariants sous cette action 
sont dits quasi-invariants, et les polynomes super-coinvariants sont les polynomes 
orthogonaux aux polynomes quasi-invariants sans terme constant (pour un certain 
produit scalaire). Notre resultat principal est I'obtention d'une base de Grobner 
pour I'ideal engendre par les polynomes quasi-invariants. Nous en deduisons alors 
que la dimension de I'espace des polynomes super-coinvariants est m" C„ oii C„ est 
le n-ieme nombre de Catalan. 



1. Introduction 

Let X denote the alphabet in n variables (xi, . . . , x„) and C[X] denote the space of 
polynomials with complex coefficients in the alphabet X. Let G„,m = Cm I Sn denote 
the wreath product of the symmetric group iS„ by the cyclic group Cm- This group 
is sometimes known as the generalized symmetric group [cf. p^). It may be seen 
as the group of n x n matrices in which each row and each column has exactly one 
non-zero entry (pseudo-permutation matrices), and such that the non-zero entries 
are m-th roots of unity. The order of Gn,m is m"n!. When m = 1, Gn,m reduces to 
the symmetric group Sn, and when m = 2, Gn,m is the hyper octahedral group Bn, 
i. e. the group of signed permutations, which is the Weyl group of type B (see [H] 
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for example for further details). The group Gn,m acts classically on C[X] by the rule 

(1.1) yg G Gn,m, VP G C[X], g.P{X) = P{X}g), 

where g is the transpose of the matrix g and X is considered as a row vector. Let 

Invn,m = {P e C[X] I V(7 G f7.P = P} 

denote the set of G„^m-iiivariant polynomials. Let us denote by Inv^^ the set of 
such polynomials with no constant term. We consider the following scalar product 
on C[X]: 

(1.2) {P,Q) = P{dX)Q{X) U=o 

where dX stands for (5xi, . . . , dxn) and X = stands for xi = ■ ■ ■ = x„ = 0. The 
space of G„ m-coinvariant polynomials is then defined by 

Covn^m = {P^ C[X] / Vg G Q{dX)P = 0} 

= {InvlJ^ ^ C[X]/{InvlJ 

where {S) denotes the ideal generated by a subset S of C[X]. 
A classical result of Chevalley [6] states the following equality: 

(1.3) dimCoVn,m = \Gn,m\ = m'^ n\ 

which reduces when m = 1 to the theorem of Artin [Ij that the dimension of the 
harmonic space H„ = CoVn,i {cf. [9]) is n!. 

Our aim is to give an analogous result in the case of quasi-symmetrizing action. 
The ring Qsym of quasi-symmetric functions was introduced by Gessel [TT] as a 
source of generating functions for P-partitions [TS] and appears in more and more 
combinatorial contexts [5l[T8l[l9]. Malvenuto and Reutenauer [16j proved a graded 
Hopf duality between QSym and the Solomon descent algebras and Gelfand et. al. [lOj 
defined the graded Hopf algebra NG of non-commutative symmetric functions and 
identified it with the Solomon descent algebra. 

In [2], [3], Aval et. al. investigated the space SH„ of super-coinvariant polynomials 
for the symmetric group, defined as the orthogonal (with respect to (11.21) ) of the ideal 
generated by quasi-symmetric polynomials with no constant term, and proved that 
its dimension as a vector space equals the n-th Catalan number: 

(1.4) dimSH„ = C„ = ^ 

Our main result is a generalization of the previous equation in the case of super- 
coinvariant polynomials for the group Gn,m- 

In Section 2, we define and study a "quasi-symmetrizing" action of Gn,m on C[X]. 
We also introduce invariant polynomials under this action, which are called quasi- 
invariant, and polynomials orthogonal to quasi- invariant polynomials, which are 
called super-coinvariant. The Section 3 is devoted to the proof of our main result 
(Theorem 12. 4p . which gives the dimension of the space SGoVn,m of super-coinvariant 
polynomials for Gn,m- we construct an explicit basis for SGoVn,m from which we 
deduce its Hilbert series. 
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2. A QUASI-SYMMETRIZING ACTION OF G^. 



We use vector notation for monomials. More precisely, for v 
we denote the monomial 



[^1 



, . . . , 



(2.1) 



For a polynomial P G Q[X], we further denote [X'^] P{X) as the coefficient of the 
monomial X'^ in P{X). 

Our first task is to define a quasi-symmetrizing action of the group Gn,m on C[X], 
which reduces to the quasi-symmetrizing action of Hivert (c/. [12]) in the case n = 1. 
This is done as follows. Let A G X he a subalphabet of X with I variables and 
K = [ki, . . . , ki) he a. vector of positive (> 0) integers. If i? is a vector whose entries 
are distinct variables Xi multiplied by roots of unity, the vector (-B)< is obtained 
by ordering the elements in B with respect to the variable order. Now the quasi- 
symmetrizing action of g E Gn m is given by 



(2.2) 
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A 



K 



w[g)<^\A}\g\). 



K 



where w{^g) is the weight of (7, i.e. the product of its non-zero entries, \g\ is the matrix 
obtained by taking the modules of the entries of (7, and the oefficient c{K) is defined 
as follows: 

n^-. 10 if Vi, A;,- = \m\ 
c{K) = 



Example 2.1. If m 

then for example 



1 if not. 



3 and n = 3, and we denote by j the complex number j 



{xl X2] 



(2,1) 



e 3 . 















if? 


{{I 





i) 


• {Xl,X2) 






1 







J [X3,Xi)^ 



(2,1) 



= f{x„xsY'''^ 

■2 2 
= J X^Xs. 

It is clear that this defines an action of the generalized symmetric group Gn,m on 
C[X], which reduces to Hivert's quasi-symmetrizing action [cf. [12\, Proposition 3.4) 
in the case m = 1. 

Let us now study its invariant and coinvariant polynomials. We need to recall some 
definitions. 

A composition a = (ai, 02, • • • , «fc) of a positive integer d is an ordered list of 
positive integers (> 0) whose sum is d. For a vector v G N", let c(v) represent the 
composition obtained by erasing zeros (if any) in v. A polynomial P G Q[A] is said 
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to be quasi- symmetric if and only if, for any u and /i in N", we have 

[X'']P{X) = [X^]P{X) 

whenever c(z/) = c(/i). The space of quasi-symmetric polynomials in n variables is 
denoted by Qsym^. 

The polynomials invariant under the action fl2.2p of Gn.m said to be quasi- 
invariant and the space of quasi-invariant polynomials is denoted by QInVn,m, i-^- 

P G QInVn,rn ^ G Gn,m, Q • P = P. 

Let us recall (c/. [12], Proposition 3.15) that QInVn,i = QSymn- The following 
proposition gives a characterization of QInVn,m- 

Proposition 2.2. One has 

P G Qlnvr,,m ^ 3g G g5ym„ / P(X) = g(X'") 
where Q(X") = Q^x"^, • • • 

Proof. Let P be an element of QInVn,m- Let us denote by C, the m-th root of unity 
= e"^ and by (7j the element of Gn,m whose matrix is 

/C \ 
1 



V 1 / 

with the C in place j. Then we observe that the identities 

Vj = l,...,n, -[P + g^.P + g^.P+... + g^~^.P) = P 

imply that every exponents appearing in P are multiples of m. Thus there exists 
a polynomial Q G C[X] such that P{X) = Q{X"^). To conclude, we note that 
Sn C Gn,m implies that P is quasi-symmetric, whence Q is also quasi-symmetric. 
The reverse implication is obvious. □ 

Let us now define super- coinvariant polynomials: 

SCoVn,m = {Pe C[X] I Vg G Qlnvn,^, Q{dX)P = 0} 
= {QlnvlJ^ ^ C[X]/{QInv+J 

with the scalar product defined in fll.2p . This is the natural analogous to CoVn in 
the case of quasi-symmetrizing actions and SCoVn,m reduces to the space of super- 
harmonic polynomials SH„ (c/. [3J) when m = 1. 

Remark 2.3. It is clear that any polynomial invariant under (12.21) is also invariant 
under (11. ip . i.e. Invn,m C QInVn,m- By taking the orthogonal, this imphes that 
SCoVn,m C CoVn,m- These observations somewhat justify the terminology. 

Our main result is the following theorem which is a generalization of equality (II. 4p . 
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Theorem 2.4. The dimension of the space ScoVn,m is given by 

1 /2r?\ 

(2.3) dimSCoVnm = m''Cn = m'' 

n + l\n J 

Remark 2.5. In the case of the hyperoctahedral group Bn = Gn,2, C.-O. Chow [7j 
defined a class BQSym{xo, X) of quasi-symmetric functions of type B in the alphabet 
{xq, X). His approach is quite different from ours. In particular, one has the equality: 

BQSYm{xo, X) = QSym{X) + QSym{xo, X). 

In the study of the coinvariant polynomials, it is not difficult to prove that the 
quotient C[xo, X]/ {BQSym~^) is isomorphic to the quotient C[X]/ {QSym~^) studied 
in To see this, we observe that if Q is the Grobner basis of {QSym^) constructed 
in [3] (see also the next section), then the set {a;o, ^} is a Grobner basis (any syzygy 
is reducible thanks to Buchberger's first criterion, cf. [8]). 

The next section is devoted to give a proof of Theorem 12.41 by constructing an 
explicit basis for the quotient C[X]/ {Qlnv^^. 



3. Proof of the main theorem 

Our task is here to construct an explicit monomial basis for the quotient space 
C[X]/((5/nw+^). Let us first recall {cf. [5) the following bijection which associates 
to any vector z/ G N" a path 7r(z/) in the N x N plane with steps going north or east 
as follows. If = (i/i, . . . , z/„), the path 7r(z/) is 

(0, 0) (i/i, 0) (z/i, 1) (z/i + z/2, 1) ^ {ui + 2) -> ■ ■ ■ 

^ (z^i H V Pn.n- I) —> h I'n.n). 

For example the path associated to z/ = (2, 1, 0, 3, 0, 1) is 





























J 























































































We distinguish two kinds of paths, thus two kinds of vectors, with respect to their 
"behavior" regarding the diagonal y = x. If the path remains above the diagonal, we 
call it a Dyck path, and say that the corresponding vector is Dyck. If not, we say 
that the path (or equivalently the associated vector) is transdiagonal. For example 
T] = (0, 0, 1, 2, 0, 1) is Dyck and e = (0, 3, 1, 1, 0, 2) is transdiagonal. 
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ri E 

We then have the following result which generalizes Theorem 4.1 of [3J and which 
clearly implies the Theorem 12.41 

Theorem 3.1. The set of monomials 

Bn,m = iT{r]) IS a Dyck path, < Oi < m} 

is a basis for the quotient C[Xn]/ {Qlnv:^ ^) . 

To prove this result, the goal is here to construct a Grobner basis for the ideal 
J7n,m = {QI^fjt,m)- ^e shall use results of [21 |3J. 
Recall that the lexicographic order on monomials is 

(3.1) X'^>iexX^ iff Z/>le./i, 

if and only if the first non-zero part of the vector u — fi is positive. 

For any subset S of Q[X] and for any positive integer m, let us introduce S"^ = 
{P{X"^) , P G S}. If we denote by G{I) the unique reduced monic Grobner basis 
(c/. [8j) of an ideal /, then the simple but crucial fact in our context is the following. 

Proposition 3.2. With the previous notations, 

(3.2) GiiS"")) = G{{S)r. 

Proof. This is a direct consequence of Buchberger's criterion. Indeed, if for every pair 
g, g' in G{{S)), the syzygy 

S{g,g') 

reduces to zero, then the syzygy 

S{g{Xn,g'{Xn) 

also reduces to zero in G{{S"^)) by exactly the same computation. □ 

Let us recall that in [2] is constructed a family Q of polynomials Ge indexed by 
transdiagonal vectors e. This family is constructed by using recursive relations of the 
fundamental quasi-symmetric functions and one of its property (c/. p]) says that the 
leading monomial of Gs is: LM{Ge) = X'^. Since ^ is a Grobner basis of J7n,i, the 
following result is a consequence of Propositions 12.21 and 13.21 

Proposition 3.3. The set is a Grobner basis of the ideal Jn,m- 

To conclude the proof of Theorem 13.11 it is sufficient to observe that the mono- 
mials not divisible by a leading monomial of an element of i.e. by a X"^^ for e 
transdiagonal, are precisely the monomials appearing in the set Bn,m- 
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As a corollary of Theorem 13.11 one gets an explicit formula for the Hilbert series 

(k) 

of SCoVn,m- For A; G N, let SCoVn,rn denote the projection 
(3.3) SCov'^J^l = SCovn,m n 



where Q'-'^-* [X] is the vector space of homogeneous polynomials of degree k together 
with zero. 

Let us denote by Fn^m{t) the Hilbert series of SCoVn,m, i-e. 



(3.4) F„,„(t) = dim SCov^^l 



k>0 

Let us recall that in [3j is given an explicit formula for FnX- 

(3.5) ^„,w.f„w.g^ (":*)«' 

k=0 ^ ' 

using the number of Dyck paths with a given number of factors ( c/. [T3] ) . 
The Theorem 13.11 then implies the 

Corollary 3.4. With the notations of Ii3.5\} . the Hilbert series of SCoVn,m is given 
by 

1 -t"^ 

Pn,mij^^ ~Fn{t ) 

from which one deduces the close formula 



_ i^-t) - Vi^-m - t - ^t^xjl - t-)) - 2x(l-t-) 
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